A Process Model of Cased-Based Reasoning in Problem Solving

Much of the problem so lv ing done by both novices and experts uses "case-based" reasoning, or reasoning by analogy to previous s i m i l a r cases. We explore the ways in which case-based reasoning can help in problem s o l v i n g . According to our model, t r ans fe r of knowledge between cases is guided l a r ­ gely by the problem so lv ing process i t s e l f . Our model shows the i n t e r a c t i o n s between problem s o l ­ v ing processes and memory fo r exper ience. Our com­ puter program, ca l l ed the MEDIATOR, i l l u s t r a t e s case-based reasoning in i n t e r p r e t i n g and reso lv ing common sense d i spu tes . 1. USING PREVIOUS CASES IN PROBLEM SOLVING Most approaches to problem so lv ing in Al t r ea t each problem as a unique case. One de f i c i ency of such problem solvers is tha t they cannot r e l y on previous exper ience. The same problem presented two d i f f e r e n t times requ i res the same (possib ly long and complex) set of reasoning s teps . Nor can such systems learn from the mistakes they have made. Suppose, however, tha t our systems could augment t h e i r problem so lv ing c a p a b i l i t i e s by making analogies to previous s i m i l a r cases. Reference to a previous case focuses reasoning on those par ts of a cu r ren t problem which were im­ por tan t in ana lys is of the previous case, poten­ t i a l l y reducing the number of fea tures tha t need to be considered. When the process of reaching a s o l ­ u t i o n to a problem involves many s teps, analogy to a previous case can o f ten reduce the number of steps requ i red to reach a s o l u t i o n . Furthermore, i f the set of cases used in problem so l v ing include those where e r ro rs had been detected and l a te r f i x e d , then analogy to a previous case can not only help in recover ing from s i m i l a r e r r o r s , but a lso in avo id ing s im i l a r f a i l u r e s in the f u t u r e . The problem so l v i ng process which solves problems by analogy to previous or hypo the t i ca l cases is a "case-based reasoning" process. Ap­ p l i c a t i o n o f previous experiences in problem s o l ­ v ing is something people seem to do in areas as d iverse as mathematics, phys ics , medic ine, and law * This research has been supported in par t by NSF Grant Nos. IST-8116892 and IST-8317711 and in par t by the A i r Force I n s t i t u t e of Technology. The views expressed are so l e l y those of the au thors . ( e . g . , Gick and Holyoak, 1980; Polya, 1945; Reed and Johnson, 1977; R iss land , 1982). It plays an espec ia l l y important r o l e in learn ing new tasks ( e . g . , Anderson, et a l . 1984; Ross, 1982), and is an important par t of making p red i c t i ons in unders­ tanding (Schank, 1982) . The example below i l l u s t r a t e s several places where analogy to a previous case can help in under­ standing and so lv ing a cur rent one. A reasoner reads about the Sinai d ispute (before the Camp David Accords). She is reminded of the Korean War since both are over land, both are compe t i t i ve , in ne i ther can the c o n f l i c t be resolved completely fo r both s ides , and in bo th , m i l i t a r y force had been used previous to n e g o t i a t i o n s . Based on t h i s remin­ d i n g , she p red ic t s that Is rae l and Egypt w i l l d i v i d e the Sinai equa l l y . She la te r reads tha t t h i s advice was given and re jec ted by both s ides . She is reminded of her daughters ' quarre l over an orange. She had suggested tha t they d i v i d e i t equa l l y , and they had re jec ted t h a t , s ince one wanted to use the e n t i r e peel fo r a cake. Rea l iz ing tha t she hadn ' t taken t h e i r real goals i n to account, she then suggested tha t they d i v i d e i t agreeably by par ts — one tak ing the pee l , the other the f r u i t . This provides the suggest ion tha t f a i l ­ ures may occur because the goals of the d isputants are misunderstood. She there fo re a t ­ tempts a r e i n t e r p r e t a t i o n of I s r a e l ' s and Egypt 's goa ls , and decides tha t Is rae l wants the Sinai as a m i l i t a r y bu f fe r zone in support of na t iona l s e c u r i t y , and Egypt wants the land back fo r na t iona l i n t e g r i t y . Further reasoning aided by analogy to the orange d ispute leads to the conclus ion tha t an "agreeable d i v i s i o n " based on the real goals of the d ispu tan ts is app rop r i a te . She is reminded of the Panama Canal d ispute s ince the d i s p u t a n t s , d isputed o b j e c t , p a r t i c i p a n t goa ls , and se lected r e s o l u t i o n plan are s i m i l a r to those in the cu r ren t d i s p u t e . Analogy to tha t inc iden t guides ref inement of the "agreeable d i v i s i o n " p l an . Replacing the US by Is rae l (the par ty c u r r e n t l y in con t ro l of the ob ject ) and Panama by Egypt (the par ty who used to own it and wants it back) , she p red i c t s tha t Egypt w i l l get economic and p o l i t i c a l con­ t r o l o f the S i n a i , wh i l e i t s normal r i g h t o f m i l i t a r y con t ro l w i l l be den ied. As the example shows, case-based reasoning can a id in i n i t i a l understanding of a problem; in genera­ t i o n of s o l u t i o n s ; and in cases of misunderstanding J. Kolodneretal. 285 or plan f a i l u r e , in r e i n t e r p r e t a t i o n and se lec t i on of a l t e r n a t e l i nes of reasoning. A problem solver must have several c a p a b i l i t i e s to employ case-based reasoning. I t must locate previous s im i l a r cases in a p o t e n t i a l l y large long term memory of past problem so lv ing episodes. It must in tegra te new cases in to memory fo r f u tu re use. It must determine whether or not a reca l l ed case is ap­ p l i c a b l e in so lv ing the new problem, and if many previous cases are a v a i l a b l e , which are p o t e n t i a l l y the most a p p l i c a b l e . F i n a l l y , i t must t rans fe r knowledge c o r r e c t l y from the previous case to the cur rent one. Much work is being done c u r r e n t l y in the area of making analogies across domains ( e . g . , Bu rs te in , 1983; Centner, 1982; Winston, 1980), mostly for making automatic knowledge a c q u i s i t i o n of a new domain eas ie r . In genera l , the analogous concept is given and the computer's job is to do the t r a n ­ s fer c o r r e c t l y . Our goals are somewhat d i f f e r e n t . Though analogy to a previous s im i l a r case resu l t s in a c q u i s i t i o n of new or more re f ined knowledge in a domain the system already knows about, we focus on analogy's ro les dur ing problem s o l v i n g . We thus consider the requirements it places on the problem so lv ing system and the cons t ra in t s put on the analog ica l process by the task demands of the problem so l ve r . Since we inves t iga te a process tha t is meant to be par t of a problem so lve r , we requ i re tha t the computer loca te , choose, and f i g u r e out how to apply appropr ia te analogies by i t s e l f . II THE CASE-BASED REASONING PROCESS The case-based reasoning process works as f o l ­ lows: (1) Locate and r e t r i e v e p o t e n t i a l l y ap­ p l i c a b l e cases from long term memory. (2) Evaluate selected cases to determine the app l i cab le ones. (3) Transfer knowledge from the o ld case (s) to the cur ren t one. We begin by exp la in ing memory s t ruc tu res and processes that a l low previous cases to be remembered. We then consider the ro les those cases can play in problem so lv ing and how problem solver task domains guide t rans fe r of knowledge from one case to another. We consider the types of memory schemata necessary to support our model, and the choice of cases when several are a v a i l a b l e . F i n a l l y , we present our program, the MEDIATOR, which uses analogy to previous cases to resolve d i spu tes . For more in format ion about other aspects of the process, see (Kolodner & Simpson, 1984 or Simpson, 1985)• IIl. LOCATING CASES IN MEMORY In order to use previous experience, a problem solver must be able to i n t e rac t w i th a memory for experience (Kolodner & Simpson, 1984) . The memory we use organizes experiences (cases) based on genera l ized episodes (Kolodner, 1984, Schank, 1982). These s t ruc tu res hold general ized knowledge desc r ib ing a c lass of s im i l a r episodes. An i n ­ d i v i dua l experience is indexed by features which d i f f e r e n t i a t e i t from the norms of the c lass (those features which can d i f f e r e n t i a t e i t from other s i m i l a r exper iences) . As a new experience is i n ­ tegrated i n to memory, i t c o l l i d e s w i th other ex­ periences in the same general ized episode which share i t s d i f f e r e n c e s . We c a l l such a c o l l i s i o n a " remind ing" (Kolodner, 1984, Schank, 1982). This t r i g g e r s two processes. Expectat ions based on the f i r s t episode can be used in analys is of the new one (analogy). S i m i l a r i t i e s between the two episodes can be compiled to form a new memory schema w i t h the s t r uc tu re j u s t descr ibed (gene ra l i za t i on ) . Figure 1 shows how the "orange d i spu te " (from the example above) is organized in a genera l ized episode associated w i th "phys ica l d ispu tes" ( i . e . , those over possession of a physical o b j e c t ) . At the top of the f i g u r e are the norms for "phys ica l d i spu tes . " Below t h a t , the "orange d i spu te " is i n ­ dexed by those features which spec ia l i ze it in the "phys ica l d i spu te " general ized episode. I t i s i n ­ dexed by features of both of i t s i n t e r p r e t a t i o n s . As new cases are added to the memory s t r u c t u r e , sub categor ies w i l l be created in the s t r u c t u r e at the places in which other cases correspond to t h i s one. The "orange d i spu te " w i l l help form the basis of norms associated w i th those ca tegor ies . Ret r ieva l of cases from memory and i n t e g r a t i o n of new cases in to memory are accomplished through a t raversa l process. As a new case is processed, ap­ p rop r ia te general ized episodes are i d e n t i f i e d . Features d i f f e r e n t i a t i n g the new case from others organized in the same category are used to c rea te new ind ices . Indices associated w i t h fea tu res already present in the indexing s t r u c t u r e are t raversed. Any previous cases the new case c o l ­ l ides w i th are ava i l ab le for f u r t he r eva lua t i on . A previous experience can thus be "remembered" if i t is organized in the same general ized episode a new case is being in tegrated in to and a lso shares a set of d i f fe rences w i t h that case. The "orange d ispu te" can be reca l l ed from the "phys ica l d i s p ­ u tes" general ized episode shown above by any physical d ispute for which " d i v i d e equa l l y " f a i l s ( e . g . , the Sinai d i s p u t e ) , any time the ob jec t of d ispute is e d i b l e , e t c . Memory processes are in tegrated w i th problem so lv ing processes as f o l l o w s : As a case is being considered, it is represented by a set of schemata (frames) which d e t a i l features of the case ( in the s l o t f i l l e r s ) . At any moment, memory has a v a i l a b l e to it the cur rent problem rep resen ta t ion . A memory t raversa l process running concur rent ly w i t h problem so lv ing processes uses the cur ren t problem representa t ion as a key in to memory. Cases the memory encounters in t ravers ing memory using the current case as i t s search key are a v a i l a b l e f o r case-based reasoning. As the problem representa­ t i o n changes, search is d i rec ted to places in memory referenced by the most cu r ren t problem represen ta t ion . IV. CASE-BASED REASONING'S VARIED ROLES Before d iscussing the ro les case based reasoning can play in problem s o l v i n g , we need to duscuss the problem so lv ing framework i n to which we enter our case based processes. In order to use case-based reasoning, a problem solver must be able to receive and evaluate feedback about the r e s u l t s of i t s dec is ions . A problem solver that suggests so lu t ions to problems but never knows the outcome of i t s advice has no basis fo r eva lua t ing i t s dec is ions , and thus cannot be expected to use i t s experience r e l i a b l y in dea l ing w i t h l a t e r cases. This suggests tha t the problem solver must have fo l low-up procedures, inc lud ing procedures f o r recogn iz ing , e x p l a i n i n g , assigning blame, and a t ­ tempting to co r rec t f a i l u r e s . In a d d i t i o n to t ha t and plan generat ion procedures normal ly included in 286 J. Kolodneretal. problem so l ve rs , we a lso include a problem unders­ tanding phase in which a p a r t i a l l y s p e c i f i e d problem d e s c r i p t i o n is e laborated to the extent necessary for problem r e s o l u t i o n . These processes are depicted in Figure 2. Case-based reasoning can play a ro le in any of these tasks . PHYSICAL DISPUTES A. Case-based reasoning in i n t e r p r e t a t i o n Before a problem can be so lved, it must be un­ ders tood. De ta i l s not presented in the problem d e s c r i p t i o n must be f i l l e d in through inference or query, and schemata (general ized episodes) fo r represent ing the problem must be chosen. Problem schemata po in t to p o t e n t i a l s o l u t i o n plans and thus are c r u c i a l to d e r i v i n g good s o l u t i o n s . Analogy to a previous case can d i r e c t these processes in the f o l l ow ing two ways: 1. by suggesting add i t i ona l fea tures to be inves t iga ted 2. by suggesting a l t e r n a t i v e i n t e r p r e t a t i o n s The f i r s t step in understanding a problem is to make a hypothesis about a schema that might descr ibe the problem. A doc tor , for example, may make a hypothesis about the d isorder a pa t i en t has by focussing on reported symptoms. Af te r represen­ t a t i o n hypotheses are made, case-based reasoning can help in v e r i f y i n g them. An attempt is made to in tegra te the new case in to the hypothesized genera l ized episodes, causing remindings. I f the cur ren t case is missing d e t a i l s necessary to v e r i f y a hypothesized schema, and a remembered case i n c l ­ udes those d e t a i l s , an attempt is made to t rans fe r those d e t a i l s to the cur ren t case. I f t he i r assum­ p t i on is cons is ten t w i t h the cur ren t case, they are f i l l e d i n , and the re levant hypothesis (the schema associated w i th the remembered case) is v e r i f i e d . A case one is reminded of may a lso have had features important to i t s proper r e s o l u t i o n tha t were not p red ic ted by the schema. These fea tures are i nves t i ga ted . On the other hand, t rave rsa l may r e s u l t in reminding of a case tha t i n i t i a l l y was thought to be represented by the hypothesized schema, but was la te r found to need a d i f f e r e n t rep resen ta t i on . In t h i s s i t u a t i o n , the cur ren t case is evaluated to see whether it might a lso be represented the second way, thus avoid ing the previous i n t e r p r e t a t i o n e r r o r . B. Case-based reasoni ng in plan generat ion Af ter a case is understood, a plan for i t s r e s o l u t i o n must be generated. Our approach, which we re fe r to as plan i n s t a n t i a t i o n , cons is ts of two major stages: p lan s e l e c t i o n and plan re f inement . Plan s e l e c t i o n involves choosing the best from among a set of known plans or cons t ruc t i ng a plan by combining appropr ia te pieces of known p lans . Plan ref inement involves r o l e b ind ing and ad jus t i ng the plan fo r the p a r t i c u l a r s i t u a t i o n . When several plans are f e a s i b l e , the expected r e s u l t s of using the plan must be generated and evaluated to choose the best one. Analogy to previous cases can help plan generat ion in four ways: 3. suggest ion of procedures to be fo l lowed 4. suggest ion of procedures to be avoided 5. s e l e c t i o n of a means of implementing a plan 6. p r e d i c t i o n of the outcome of a se lected plan Plans associated w i t h cases remembered dur ing i n t e r p r e t a t i o n are a v a i l a b l e to suggest plans to be fo l lowed or avo ided. The precond i t ions of any suc­ cessfu l p lan are checked, and i f a p p l i c a b l e , the plan is a t tempted. At the same t ime, any plan used J. Kolodner et al . 2 8 7 i n a p r e v i o u s case and r e s u l t i n g i n f a i l u r e i s p r o h i b i t e d i n t h e c u r r e n t c a s e . I f more t h a n one p l a n i s s u g g e s t e d , t h e u t i l i t y o f p o t e n t i a l p l a n s must b e e v a l u a t e d and t h e most a p p r o p r i a t e one chosen . P lan e v a l u a t i o n i n v o l v e s s i m u l a t i n g t h e r e s u l t s o f a l t e r n a t i v e c o u r s e s o f a c t i o n and e v a l u a t i n g them. E v a l u a t i o n can b e done t a k i n g p r e v i o u s e x p e r i e n c e s i n t o a c ­ c o u n t . S i m u l a t i n g t h e r e s u l t s o f u s i n g a p l a n p r o v i d e s a h y p o t h e t i c a l s i t u a t i o n wh i ch may be. s i m i l a r t o a p r e v i o u s r e a l o n e . The r e s u l t s o f p r e v i o u s a t t e m p t s a t i m p l e m e n t i n g t he same p l a n under s i m i l a r c o n d i t i o n s p r o v i d e a way o f e v a l ­ u a t i n g a p o t e n t i a l c o u r s e o f a c t i o n . A d d i t i o n a l i n f o r m a t i o n l e a d i n g t o a b e t t e r s t r a t e g y may a l s o b e p r o v i d e d d u r i n g e v a l u a t i o n . T h i s p r o c e s s i s a n e x t e n s i o n o f what Schank (1982) r e f e r s t o as i n t e n ­ t i o n a l r e m i n d i n g , and i s a component o f W i l e n s k y ' s (1983) P r o j e c t o r . Any chosen p l a n must be r e f i n e d or a d j u s t e d f o r t he c u r r e n t s i t u a t i o n . A s u i t a b l e p l a n im ­ p l e m e n t a t i o n can be d e r i v e d by c o n s i d e r i n g now i t was e f f e c t e d in a p r e v i o u s c a s e . We see t h i s in t h e S i n a i example when the Panama Canal agreement i s used i n d e c i d i n g how t o c a r r y o u t t h e " d i v i d e a g r e e a b l y " p l a n . C. Case based r e a s o n i n g i n e r r o r r e c o v e r y Prob lem s o l v i n g e r r o r s u s u a l l y appear a s p l a n f a i l u r e s , b u t t hey can r e s u l t f r om i n i t i a l m i s i n ­ t e r p r e t a t i o n , poor i m p l e m e n t a t i o n o f t h e p l a n , i n c o r r e c t p r e d i c t i o n o f t h e r e s u l t s , new unexpec ted o c c u r r e n c e s , o r bad p l a n s e l e c t i o n . When s i m i l a r f a i l u r e s have happened p r e v i o u s l y , t h e work i n v o l v e d in e r r o r r e c o v e r y can be c u t down. In a s e n s e , e r r o r r e c o v e r y can be v iewed as ano the r i n s ­ t a n c e o f p r o b l e m s o l v i n g , t h i s t i m e i n t e r p r e t i n g t h e f a i l u r e ( e x p l a i n i n g i t ) and f i x i n g t h e f a u l t y knowledge ( r e m e d i a t i o n ) . A p r e v i o u s case can t h u s p l a y r o l e s s i m i l a r t o t hose p l a y e d i n i n i t i a l i n ­ t e r p r e t a t i o n and p l a n n i n g : 7 . s u g g e s t i o n o f a n e x p l a n a t i o n f o r t h e